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Re: RFP No. 40-RQ20680730: Read to Achieve Diagnostics - Software Av A Service 


Follow-up from Protest Meeting 


Dear Ms. Dunn and Mr. Thomas: 

On behalf of Amplify Education, Inc. (“Amplify”), I wanted to thank the NC Department 
of Public Instruction (“DPI”) for allowing us the opportunity to have a protest meeting on the 
afternoon of July 18, 2019. We appreciated the attention of the many DPI staff who joined us in 
the Superintendent’s conference room. Since Jonathan Sink led the meeting, but has departed DPI, 
I am copying Mr. Thomas in his place as counsel for DPI. 

As was noted in Amplify’s presentation, a number of the points and concerns raised in our 
original protest letter were strongly validated when DPI released 166 pages of public records on 
the afternoon of Friday July 12 (referenced herein as “PDF” with page references). 
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I. Amplify’s proposal was clearly determined to be superior by DPI, and IStation was 

deemed not to adequately satisfy critical statutory and RFP requirements. 

Amplify’s CEO Larry Berger explained in the meeting that the company’s position that 
mCLASS is indeed the best offering for the state of North Carolina was confirmed by the detailed 
work done by the original Evaluation Committee. You do not need to merely take Amplify’s word 
for it. The Evaluation Committee ranked Amplify #1 among all proposals on the three most 
important evaluation criteria of the procurement. On one after another of the required 
specifications that were tagged as “legislatively mandated” by DPI, Amplify received a unanimous 
11-vote “yes” of the Committee for meeting the RFP’s requirements, when for many of the same 
specifications IStation received zero “yes” votes. This included the RFP requirement of dyslexia 
screening 1 , identified as one of the two purposes of the procurement by DPI at the outset of the 
evaluation 2 . Zero committee members believed IStation met the state’s requirements for dyslexia 
screening. This was further confirmed in the Evaluation Ranking materials where it is stated that 
“Istation is not a reliable screener for dyslexia because it lacks some key measures for dyslexia 
risk factors .... This will require multiple tools for assessment.” 3 Indeed, even in the final (and 
much less detailed) DPI memo of June 7, 2019 intended to justify the award to IStation, DPI 
admitted that IStation’s product is “missing key measures” for testing dyslexia. (PDF p. 28 of 
166.) This reflects the concern of the original Evaluation Committee that districts would have to 
use two assessment tools to meet the statutory requirements for dyslexia testing if IStation had 
been their choice. 

DPI’s public records further show that DPI agreed (as we stated in our protest letter) that 
Amplify’s offering met the State’s requirement that an offering be “developmentally appropriate” 4 
for the K-3 student population. According to the Evaluation Consensus Meeting Notes from 
November 19, 2018, no Evaluation Committee members believed that IStation met the 
“developmentally appropriate” requirement, and seven affirmed that IStation did not meet it. As 
the Committee stated: “Some of the members were concerned about the all online piece of this 
assessment and how appropriate it is for Kindergartners and struggling learners.” 5 


1 The RFP required a solution that could “adequately and accurately identify indicators of risk for 
dyslexia in grades K-3 as outlined in NC Session Law 2017-127,” RFP, Spec. 8, at p. 26. 

2 See page 3 of the October 5, 2018 Proposal Evaluation KickOff Meeting, PDF p. 79 of 166. 

3 PDFp. 42 of 166. 

4 RFP, Spec. 9 at p. 26. 

5 PDF p. 59 of 166. “The team voted 7 No and 4 Maybe for this one as to IStation. Some voting 
members were concerned about the all online piece of this assessment and how appropriate it is 
for Kindergarteners and struggling learners. The assessment was also for 40 minutes.” According 
to page 26 of the October 5, 2018 Proposal Evaluation Kick Off Meeting, approvals from all voting 
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Public records show that in determining whether the vendors demonstrated “Substantial 
Conformity to Solicitation Specifications,” the Evaluation Committee ranked Amplify higher in 
the majority of the requirements identified as “legislatively mandated” and deemed most critical 
by DPI: 

Excerpt of table presented by Amplify at protest meeting (full copy attached to this letter as 
Exhibit A): 

Evaluation Consensus 

Requisition No. RQ20680730 (Meeting Notes, November 19-20, 2018) 


Committee Votes 

Amplify 

Istation 

REQUIREMENTS 

YES VOTES 

YES VOTES 

Dyslexia Screening (BS9) 

11 . 

0 ZERO YES VOTES 

Developmental^ Appropriate BS8) 

11 . 

0 ZERO YES VOTES 

Information Security Audit SOC2 (TS35) 

11 . 

0 ZERO YES VOTES 

Reading Assess To Inform Instruction BS1) 

11 . 

0 ZERO YES VOTES 

Standards Alignment ( BS17) 

7 . 

0 ZERO YES VOTES 

Progress Monitoring (BS6) 

8 . 

1 * 

Diagnostic Assessment (BS12) 

10 . 

2 •• 

Project Management /WBS (PMS1) 

11 . 

2 •• 


As we also detailed in the protest meeting, there were also at least two fatal procedural 
flaws in the procurement process, both of which require the award to IStation to be voided. 


members were required on meeting minutes from the consensus meetings, such as the one on 
November 19. (PDF p. 102 of 166.) 
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I. The “cancellation” of the RFP was done after the proposals and costs of each had 

already been evaluated, yet PIT was told DPI could not evaluate the costs. 

First, the “cancellation” of the RFP was clearly improper. In December 2018 and January 
2019, DPI’s public records production (which was represented last week as being every record 
known to the agency at this time on the procurement) reflects that the Evaluation Committee 
ranked Amplify as the clear number one proposal among the four bidders, and that IStation’s 
product was found to be deficient and failed to meet key RFP requirements. By January 2019, the 
Evaluation Committee had reviewed all four proposals in depth and evaluated them on every one 
of the RFP’s criteria, including cost. 

After this full evaluation of all the bids, in a March 2019 meeting of the Evaluation 
Committee, the minutes of that meeting (PDF p. 13 of 166) state: 

One of the voting members breached the confidentiality provisions of the 
procurement process which jeopardized the legality of this procurement. It should 
also be noted that the team did not reach a unanimous consensus on the choice of 
the finalists vendors. Because of these issues the current read to achieve 
procurement has to be cancelled again. 

This is DPI’s basis for the cancellation of the RFP. Yet, DPI’s public records do not 
document any breach of any confidentiality that occurred on the 2018 RFP, and if any such breach 
occurred, no records document how that affected the integrity of the procurement. 6 In the protest 
meeting, DPI made a reference to a breach of confidentiality but did not disclose any information 
about it and simply referred Amplify to the public records production, which contains only the 
information quoted above from the March 2019 meeting. Mr. Sink confirmed in the meeting that 
he does not possess and is not aware of any public records that document any such confidentiality 


6 Indeed, as Amplify noted at the protest meeting, DPI spokesperson Graham Wilson apparently 
stated (mistakenly) just last week that an evaluation committee member had a past business 
relationship Amplify that was undisclosed, and that this was the alleged basis for the cancellation 
referenced in the public records. However, Wilson later corrected that statement, as it turned out 
that was an issue raised in the 2017 RFP that was cancelled, not the 2018 RFP. The allegation 
apparently concerned someone who was a contractor responsible for scheduling trainings and other 
logistical details for a few months for Amplify in 2011/2012, and never had anything to do with 
the substance of Amplify’s offerings. But DPI admits this matter had nothing to do with the 2018 
RFP cancellation. 

See www.newsobserver.com/news/business/article232712817.html where this information and 
correction was published in the News & Observer. 
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breach, other than the mysterious reference in the March 2019 meeting minutes. DPI apparently 
has no records documenting any alleged confidentiality breach that required a cancellation. 

On the “unanimous consensus” point of the March 2019 minutes, Mr. Sink agreed with 
Amplify in the protest meeting that the normal meaning of the term “consensus” does not mean 
“unanimous.” It sounds like DPI believes DIT told them that “consensus” requires unanimity, but 
none of this is documented in any public records. As the original Evaluation Committee noted in 
its November 19, 2018 meeting minutes, “consensus means general agreement and not unanimity.” 
(PDF p. 46 of 166.) But even then, the Evaluation Committee’s November 2018 notes show that 
they did reach a unanimous decision that Amplify’s proposal was the best ranked proposal. (PDF 
p. 71 of 166.) Nothing in DPI’s records prior to the March 2019 meeting minutes reflect that the 
Evaluation Committee was unable to make a vendor selection. 

As you know, DPI had to get permission from DIT to cancel this information technology 
procurement. 06 NCAC 06B.04041. The public records produced by DIT and DPI show that DIT 
understood that DPI sought a cancellation because the cost proposals of Amplify and IStation “are 
at a disparate price range (so unlike that there is no basis for comparison).” (DIT Approval Form 
for Cancellation, March 25, 2019, Exhibit B). At the protest meeting you also pointed us to a 
March 21, 2019 memo from DPI to DIT requesting a cancellation which provides no justification 
for cancellation, and does not disclose the reasons for cancellation documented in the March 2019 
Evaluation Committee minutes, attached as Exhibit C. The basis for cancellation conveyed to 
DIT does not match DPI’s basis for cancellation. Indeed, there are no records reflecting that DIT 
was told that DPI had already reviewed all the proposals fully and had evaluated the costs of each 
proposal and ranked them on cost (and every other evaluation criteria.) Thus, any claim that DPI 
could not evaluate the cost differential of the proposals is wrong, and not a valid basis for 
cancellation of the RFP process. 7 

Therefore, the cancellation of the RFP in March 2019 was a flawed decision and cannot 

stand. 

II. DPI changed the evaluation criteria and ranking of importance of criteria after 
reviewing all the proposals. Incredibly, this looks like a blatant attempt to simply 
give the contract to IStation. 

Even assuming the “cancellation” of the RFP was appropriate, even though the 
“cancellation” never seemed to change anything in the process except throw out the original 


7 Attached to this letter as Exhibit F is a note about this memo highlighting issues around SOC 2 
Type II third party attestation requirements. 



Smith, Anderson, Blount, Dorsett, Mitchell & Jernigan, L.L.P. 

North Carolina Department of Public Instruction 
July 24, 2019 
Page 6 


Evaluation Commitee’s work, 8 a second fatal procedural flaw is that DPI changed its RFP 
evaluation criteria and reordered them to the direct benefit of IStation, after it had already 
reviewed and evaluated all the proposals. In short, after DPI rated Amplify as the 4 th best proposal 
on two of the least important RFP criteria (Vendor Cost and Vendor Financial Stability), DPI 
reordered these two criteria to be the most important criteria in its final award evaluation memo. 
DPI then invented two new criteria not listed in the RFP, and threw out the most important criteria 
of the RFP - whether the vendor met the RFP specifications and state’s specific needs for Read to 
Achieve. Only by “changing the rules of the game” was DPI able to justify an award to IStation. 

As reflected in the original Evaluation Committee’s records and the RFP, there were six 
evaluation criteria in this RFP. As the following slide from the December 4, 2018 presentation 
to Superintendent Johnson shows (PDF p. 39 of 166), the criteria were ranked in the following 
order: 


Evaluation Ranking 


/ 


<3 


Evaluation Criteria 

Amplify Education 
Inc. 

Curriculum Associates Istatioi 

NWEA 

Substantial Conformity 
to specification 

i 

3 


4 

RtAD Desired 
Specifications 

i 

3 

rV 

4 

Proof of Concept / 
Demonstration 

i 

3 

1 


3 


Vendor Cost Proposal 4 

Strength of References 1 

Vendor Financial 4 

Stability 

Overall Rank 

The evaluation criteria are 


sr Public Schools of North Carolina 




G 




relative order of importance. 


The first three of these criteria (“Substantial conformity to specifications,” “RtA Desired 
Specifications” and “Proof of Concept/Demonstration”) were deemed most important, as these 
three factors determine whether vendors can actually meet the RFP’s specifications and provide a 
product that meets North Carolina’s needs. 


8 Even DPI’s June 7 award recommendation states it “recommends award under RFP No. 40- 
RQ20680730A,” which is the allegedly “cancelled” RFP. (PDF p. 29, of 166.) 
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The last three criteria in order of importance were “Vendor Cost Proposal,” “Strength of 
References,” and Vendor Financial Stability.” 

In December 2018, Amplify was ranked #1 on the top three criteria of whether it was the 
best product. It was ranked #4 on Vendor Cost Proposal and Vendor Financial Stability, 9 but, 
again, these were among the least important criteria. 

However, in the DPI’s June 7 final award recommendation, suddenly Vendor Cost 
Proposal and Vendor Financial Stability were ranked as the #1 and #2 most important criteria on 
a new list of only four total criteria. DPI did this already knowing that it believed Amplify would 
be ranked #4 on these criteria. 

This re-ordering of the importance of the award criteria can only be seen by any rational 
observer to be a blatant attempt on someone’s behalf to give this award to IStation. But that’s not 
all DPI did. In addition, after making “Cost” and “Vendor Financial Stability” its top two criteria 
(instead of #4 and #6), DPI threw out its top three criteria completely. Instead, DPI created two 
new criteria: “formative and diagnostic assessment” and “personalized learning.” (PDF p. 20 of 
166.) It is not clear what “formative and diagnostic assessment” means, since the whole purpose 
of the RFP is to procure “valid, reliable, formative and diagnostic reading assessments.” (RFP 
p. 6.) Furthermore, “personalized learning” is not among the required specifications of the RFP. 
Instead, one of the 14 non-mandatory “desirable but not required” specifications at the end of the 
RFP’s ten pages of specifications makes a reference to “a personalized blended approach to 
assessment and learning.” Certainly Amplify was not informed by the RFP or at any point that 
this would be one of only four award criteria. 

The following graphic shows DPI’s wholesale change of the criteria from the RFP and 
original Evaluation Committee to the final award to IStation last month (copy also attached to this 
letter as Exhibit D): 


9 Amplify has contracts for work in all 50 states. Amplify does not believe it has any issue with 
financial stability. For purposes of the bid protest meeting. Amplify accepted DPI’s relative 
ranking of the proposals. Even if Amplify was appropriately ranked fourth on this criterion, it 
submits that any differential among the vendors was not substantively significant on this point, 
which is consistent with this criterion being ranked last in order of importance. 
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Evaluation Criteria: Read to Achieve Diagnostics, Requisition No. RQ20680730 


Fall 2018 Evaluation 


Amplify Istation 


Spring 2019 Evaluation 


PRIORITY 

EVALUATION CRITERIA 

RANK 

RANK 

1 

Substantial Conformity To Specification 

1 

2 

2 

RtAD Desired Specifications 

1 

2 

3 

Proof Of Concept / Demonstration 

1 

2 

4 

Vendor Cost Proposal 

4 

'1 

5 

Strength Of References 

1 

1 

6 

Vendor Financial Stability 

4 

2 | 


Overall Rank 

1 

2 


EVALUATION CRITERIA 


H 


"Personalized Learning" Criterion Listed As "Desirable But Not Required” 


1 Vendor Cost Proposal 

2 Vendor Financial Stability 


3 Formative & Diagnostic Assessment 


4 Personalized Learning 


Source: Fall 2018 Evaluation: Read to Achieve - 
2018, December 4, 2018 presentation: Spring 2019 
Evaluation: Contract Award Recommendation, June 
7, 2019. 

Note: Evaluation criteria were listed in order of 
importance in both evaluations. 


It seems obvious to any rational observer why DPI had to throw out the top three evaluation 
criteria of the RFP and the original Evaluation Committee: Amplify scored in first place on all 
three. There was no way DPI could justify an award to IStation under the RFP’s requirements. 

As I discussed in the protest meeting, DPI has even admitted in public records that it had 
changed the evaluation criteria. In a May 23 draft of its award recommendation memo, DPI stated 
that it had based its decision “on the evaluation factors determined appropriate.” 

This is in contrast to what DPI stated at the time of the RFP cancellation, when DPI 
communicated to DIT that it intended to conduct negotiations to obtain best value “based on the 
evaluation factors set form [sic] in the solicitation” and informed DIT that it will be “proceeding 
in this manner.” (March 21, 2019 memo from DPI to DIT, Exhibit C). Instead, DPI proceeded to 
disregard the top three evaluation factors of the RFP, elevate the importance of two lower 
evaluation factors it already deemed Amplify to rank low on, and add two wholly new criteria. 
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Excerpts of memo (full copy attached to this letter as Exhibit E): 

Contract Award Recommendation 


To: Andrea Pacyna 

Deputy Chief IT Procurement Officer 
Department of Information Technology 

From: Tymica Dunn 

Procurement Chief 
Department of Public Instruction 

Date: 

Subject: Contract Award Recommendation 

Read to Achieve Diagnostics - Requisition # - RQ20680730, DIT File #300042 

Section 5: Evaluation of Bid Submission 

In response to DIT No. 40-IT00115-18 - DPI RFP No. 40-RQ20680730 four (4) bid proposals were 
received. The RFP cancellation was issued on March 21, 2019 since NCDPI deemed soliciting 
formal offers via IPS again would serve no valuable purpose. Pursuant to the definition of 
"Negotiation" in 9 NCAC6B.0102, DPI conducted negotiations with two (2) of the four (4) 
vendors to maximize the State's ability to obtain best value basedontheevaluationfactors 
determined appropriate. 



Only choosing to use “evaluation factors determined appropriate” after the RFP has been 
issued, and bids have been opened and evaluated, clearly violates North Carolina law. An agency 
cannot change the rules of an RFP after the bids have been opened and evaluated. This would be 
an invitation to bias and corruption. An agency must follow the rules of its RFP, and even after a 
valid cancellation cannot materially alter the scope of the original RFP. See Humble Oil & 
Refining Co v. Bd. of Alderman, 284 N.C. 458, 467, 202 S.E.2d 129, 135 (1974) (agencies must 
follow their own rules); 09 NCAC 06B.0316 (“Negotiations shall not materially alter the intent or 
scope of the original solicitation document.”) North Carolina requires competition for contracts 
for goods and services, and if the award criteria of a procurement could be changed and re-ordered 
after proposals have been opened and evaluated, an agency could simply “game the system” to 
give an award to whoever they wish. 

In the final June 7 award memo, the above reference to evaluating the proposals on “the 
evaluation factors determined appropriate” was deleted. (PDF p. 21 of 166.) Yet, DPI did not 
attempt to correct the problem. It did not reinstate the original RFP criteria and Evaluation 
Committee determinations. Instead, the award recommendation memo contains the re-ordered and 
new criteria detailed above. 
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It should also be noted that the change of evaluation criteria also violates the 2017 and 
2018 budget bills. Section 7.27(b)-(c) of the 2017 budget bill (S.L. 2017-57, p. 86-87) required 
this procurement to consider “minimum” certain factors, which were recognized by the original 
Evaluation Committee (PDF p. 37 of 166), and reflected in the detailed specifications of the 
original RFP. The RFP also contained many additional mandatory required specifications. The 
June 7 award memo does not show compliance with the RFP and budget bills as demonstrated by 
the original Evaluation Committee. The Superintendent did not have discretion in selecting an 
awardee, but had to accept the Evaluation Committee selection. S.F. 2017-57, Sec. 7.27(c). Even 
so, there is evidence in the public records that the Superintendent may have tried to influence the 
Evaluation Committee selection, telling them in a January 2019 meeting what he felt was important 
in making a selection. (PDF p. 33 of 166) 

The changing and re-ordering of the award criteria after the proposals had been opened and 
evaluated (in a way that seemed plainly designed, by someone, to reject Amplify and select 
IStation) clearly violates North Carolina procurement law. Mistakes can occur in procurements 
despite the best intentions of an agency, and the solution to this is to correct the mistake. Here, 
the award to IStation should be rescinded, and an award made to Amplify. 

A resolution to this matter is important, as another school year will soon be starting for 
North Carolina students. Some year round schools are already in session. In order for schools, 
teachers and administrators to have a successful back to school transition, with the knowledge as 
to which tool will be used for the Read to Achieve program, we encourage a timely resolution to 
this matter so that the State can limit impact to the schools and students across the state. 
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Conclusion 


Thank you again for your attention to this matter. Please let me know if you need anything 
further regarding the above mentioned materials. Pursuant to 09 NCAC 06B. 1102(c)(2) and N.C. 
Gen. Stat. § 1A-1, Rule 6(a) and N.C. Gen. Stat. § 1-593, we look forward to receiving a response 
to the protest meeting by Monday July 29. 

Very truly yours, 

SMITH, ANDERSON, BLOUNT, DORSETT, 
MITCHELL & JERNIGAN, L.L.P. 



By: J. Mitchell Armbruster 

JMA/ 

Enclosure (exhibits) 
cc: (via email) 

Eric Snider, General Counsel for SBE 

Tiffany Lucas, Special Assistant Attorney General 

Jessica Middlebrooks, Counsel for DIT 






